Quantitative studies in retinex theroy. A comparison between theoretical predictions and observer responses to the "color mondrian" experiments.

Land’s Color Mondrian experiments showed that a single wavelength-radiance distribution falling on a point on the retina can generate nearly any color sensation. In Part I we repeated that experiment. quantifying the color sensations for each of the many Mondrian areas. In Part II we show that each area‘s coior sensation correlates with a triplet of reflectances measured with photodetectors having the same spectral sensitivities as the cone pigments in the eye. This result provides a description of what the visual system does. but it does not provide a mechanism for how the visual system can do it because the reflectance measurements required the use of a reflectance standard and unchanging illumination. In Part III we describe a model for color sensations that computes three reflectances from the wavelength-radiance distribution without reflectance or illumination standards: hence. it is able to predict the color sensations seen by the observer. The model is able to predict gray. red. yellow. green and blue sensations associated with areas that send identical wavelength-radiance distributions to the eye. Ke,r Wo&--Color Mondrian experiments: color sensation: model for color sensations. In the human eye there are three types of cones, each containing a different photosensitive pigment (Marks, Dobelle and MacNichol. 1964; Brown and Wald, 1964). It is often assumed that the color at each point in the visual image depends only on the relative energies absorbed by these three pigments from the light incident at that point on the retina. In general, this simple approach to color is incorrect. While it is certainly true that any color can be matched by suitable adjustment of the intensities of three fixed primaries, a particular mixture of those primaries does not specify a unique color sensation (Wright, 1972; Wyszecki, 1973). Helmholtz (1924), in his chapter on contrast, cites a variety of observations that show that the color of an area changes when areas adjacent to it are changed. Helson (1938), Evans (1948) and Albers (1963) extended these observations experimentally. Land (1959a. 1962.1964, 1975) showed that in moderately complex images there is no unique color sensation associated with a particular wavelength-radiate distribution at a point. Land’s *‘Color Mondrian”z experiments demonstrate that a particular wavelength-radiance distribution can produce nearly any color sensation. The Color Mondrian display, used in those experiments, consisted of about 100 different colored matte papers arranged arbitrarity so that no particular color ’ Present address: Department of Physiology-Anatomy, University of California. Berkeley. CA 94720. ’ The experiment is so Wed because the visual display used in she experiment resembles a painting by Piet Mondrian. surrounded another. In fact, each paper was surrounded by at least five or six different colored papers (see Land, 1975, for color photograph of display). The display was illuminated by three projectors. each with a different ~te~erence filter. One filter transmitted part of the long-waves of the spectrum, which appear red; the second transmitted part of the middlelength waves (green): and the third, part of the shortwaves (blue). Each projector had an inde~ndent voltage control. The observers picked an area. say a white one, and the experimenter measured separately the three (long-. middleand short-wave) radiances coming from that area. Then the observer picked a second area, for example, a red one. and the experimenter measured the triplet of radiances coming from it. These measurements showed that there was slightly less long-wave light coming from the red paper than from the white, but that there was much less middleand short-wavelength light. The experimenter then adjusted the amounts of the three ~luminants so that the same triplet of long-, middleand short-wave radiances came from the red paper as came previously from the white paper. For each waveband, the experimenter increased the illumination by the factor that the white paper was a better reelector than the red paper. All three illuminants were turned on together and the observer reported that the red area still looked red. even though the radiance measurements showed that the light reaching the eye was identical to that sent by the white area a moment before. The sensation red was produced by exactly the same stimulus at a point that previously produced the sensation white. In the same manner. Land went from paper to paper in the display and produced very nearly the full gamut of color sensations with a single triplet of radiance measurements. Land proposed that something fundamental was wrong with the idea that the biological system used the physical stimulus at a point to determine color. Instead of the long-. middleand short-wave recc’ptors comparing, responses at a point. Land suggested that information from the lon_e-wave receptors was intercompared to compute a bIologica analog of retlectance from the long-uave flus. Similarlv. the information from the middle-wave receptors is ‘intercompared to form the biolo$cal analog of retlectance for that waveband and this procedure is repeated again for the short-wave receptors. This biological analog of reflectance is called lightness. The information from each of the separate sets of cones generates a separate lightness image; the comparison of three separate lightnesses for each area is the determinant of color (Land. 1964). The formation of the lightnesses and their comparison could occur in the retina or in the cortex. Experiments in visual physiology cannot as yet define the location of the interactions that must be occurring. Therefore. Land coined the word Retinex (made of “retina” and “cortex”) to designate the physiological mechanisms that generate these independent images. His proposal did not demand that the retinal elements with the same sensitivity be directly connected to each other. Instead, somewhere in the retinal-cortical structure. elements with the same wavelength sensitivity cooperate to form independent lightness images (Land. 1964). We wish to test whether the quantitative predictions of the Retinex theory match the color experience of an observer viewing the Color ,Mondrian experiments.’ This test of Retinex theory is readily divisible into three parts. In Part I we quantify the color sensations seen by the observers. We asked our subjects to choose from the !“ilunse[l Book of‘ Color the colored chips which best matched the color of each area in the Mondrian. In Part II we test whether the observers’ matches correlate with those predicted by Retinex theory. The theory states that each color is determined by a triplet of LighJltnesses. and that each lightness, in a situation like the Color Mondrian. corresponds to the reflectance of the area measured with a photodetector which has the same spectral sensitivity as one of the three cone pigments. Land’s experiments show conclusively that color sensations do not correlate with the energy at each point. Our results show that the color sensa’ We have reserved for later papers the comparison of Land’s Retinex model with other explanations of the invariance of color sensation with changes in the wavelengthradiance distribution of the light coming to the eye, such as chromatic adaptation (von Kries, 1905). The literature contains many variations of the chromatic adaptation hypothesis (Helson. 1943). a few emDirical formations such as-that of Judd (1940). and a varieiy of experiments that articulate problems with theories which assert that chromatic adaptation can account for the absence of correlation berween the wavelength-radiance distribution coming from a point and the color sensation of that point (Walters. 1942: Wassef, 1958. 1959: Land and Daw. 1962; Land. 1975). tlons are tsr> highI> correlated uith the triplets oi reflectance. The results shoi\ that the \lsual s>stem performs the analog of measuring retlectancss even though it does not LISS knou, n reflectance standards and invariant illumination. In Part III ue describe a model for calculating lightnesses from the radiances falling on sach point on the retina. The calculations required by the model were performed by computer. The inputs to the computer wtre thrse arrays of radiances measured at 480 points on the Color Mondrian display. Each array was iveighted by one of three absorption curves which characterize the cone pigments (Brolvn and Wald. 1963; Brown. unpublished). The output of the model was three arrays of computed lightnssses. We then test the model’s predictions by comparing them with the triplets of lightnesses measured from the matching chips chosen bq the observers. This comparison of observers’ choices and computer calculations shows a very Dood lit. . =